Image processing device, image processing method and recording medium

ABSTRACT

In the image processing device, the image processing method and the recording medium, the image analyzer carries out image analysis on an image. The tag information assignor assigns the image with tag information corresponding to objects present in the image based on the result of the image analysis. The first assignment ratio calculator calculates an assignment ratio of common tag information assigned to images owned by users as the first assignment ratio. The second assignment ratio calculator calculates an assignment ratio of the common tag information assigned to images owned by each user as the second assignment ratio. And the tag ranking determiner ranks the common tag information assigned to images owned by the user based on the difference between the first assignment ratio and the second assignment ratio of the common tag information.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a Divisional Application of Ser. No.15/248,570 filed Aug. 26, 2016, which claims priority under 35 U.S.C. §119 to Japanese Patent Application No. 2015-184882, filed on Sep. 18,2015. The above application is hereby expressly incorporated byreference, in its entirety, into the present application.

BACKGROUND OF THE INVENTION

The invention relates to an image processing device, an image processingmethod and a recording medium for ranking a piece of tag informationassigned to an image owned by a user.

Nowadays, there has been realized an automatic layout function thatautomatically selects a certain number of images in each of which aperson is well photographed from numerous images owned by a user, andautomatically generates a synthetic image for a calendar, an electronicalbum, a screen saver or the like using the selected certain number ofimages. The automatic layout function enables the user to easily producea synthetic image without the bother of manually selecting images fromthose owned by the user.

However, the conventional automatic layout function does not rank anon-person object present in an image and therefore cannot rank an imageincluding an object. In addition, since an object is not ranked, theconventional automatic layout function cannot rank an image in which aperson and an object are both present in consideration of not onlywhether the person is well or poorly photographed but also importance ofthe object.

Now, JP 2011-60155 A, JP 4643735 B and JP 7-287703 A are mentioned asliteratures related to the present invention.

JP 2011-60155 A describes extracting an object included in an imageprovided, identifying the extracted object, and extracting a tagcorresponding to the object.

JP 4643735 B describes determining attributes of contents by analyzingfeatures of the contents, storing information on the content andinformation showing the determined attribute of the content in amaterial information database, and calculating, with respect to each ofthe determined attributes, a sum of numbers of contents in each ofinitial groups including a content which has the attribute.

JP 7-287703 A describes obtaining the sum SUM resulting from summing ofthe intensity values of pixels, obtaining a mean value Aver from the sumSUM, and for every pixel value P, obtaining an absolute value of adifference between the pixel value P and the mean value Aver.

SUMMARY OF THE INVENTION

However, the prior art including JP 2011-60155 A, JP 4643735 B and JP7-287703 A did not have a concept of ranking a piece of tag informationassigned to an image in accordance with an object present in the imageowned by a user, that is, ranking an object corresponding to a piece oftag information.

An object of the invention is to overcome the above problem of the priorart and provide an image processing device, an image processing methodand a recording medium capable of ranking a piece of tag informationassigned to an image owned by a user in order of importance for theuser.

In order to attain the object described above, the present inventionprovides an image processing device including:

an image analyzer configured to carry out image analysis on an image;

a tag information assignor configured to assign the image with one ormore pieces of tag information corresponding to one or more objectspresent in the image based on a result of the image analysis;

a first assignment ratio calculator configured to calculate anassignment ratio of each piece of common tag information assigned toimages owned by a plurality of users as a first assignment ratio, theeach piece of common tag information being a piece of tag informationassigned to and shared by images owned by the plurality of users;

a second assignment ratio calculator configured to calculate anassignment ratio of the each piece of common tag information assigned toimages owned by each user as a second assignment ratio; and

a tag ranking determiner configured to rank the each piece of common taginformation assigned to images owned by the each user based on adifference between the first assignment ratio and the second assignmentratio of the each piece of common tag information.

Also, the present invention provides an image processing methodincluding:

a step of carrying out image analysis on an image, with an imageanalyzer;

a step of assigning the image with one or more pieces of tag informationcorresponding to one or more objects present in the image based on aresult of the image analysis, with a tag information assignor;

a step of calculating an assignment ratio of each piece of common taginformation assigned to images owned by a plurality of users as a firstassignment ratio, with a first assignment ratio calculator, the eachpiece of common tag information being a piece of tag informationassigned to and shared by images owned by the plurality of users;

a step of calculating an assignment ratio of the each piece of commontag information assigned to images owned by each user as a secondassignment ratio, with a second assignment ratio calculator; and

a step of ranking the each piece of common tag information assigned toimages owned by each user based on a difference between the firstassignment ratio and the second assignment ratio of the each piece ofcommon tag information, with a tag ranking determiner.

Further, the present invention provides a non-transitory computerreadable recording medium having recorded thereon a program for causinga computer to execute each step of the image processing method describedabove.

According to the invention, a difference between the first assignmentratio and the second assignment ratio of a piece of common taginformation is calculated, and an assignment ratio of the piece ofcommon tag information is compared between a user and other users,whereby a piece of common tag information, i.e., an object that ishighly important for each user can be determined. In this manner, apiece of tag information assigned to an image owned by a user can beranked in order of importance for the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an embodiment of the configuration ofan image processing system of the invention.

FIG. 2 is a block diagram showing an embodiment of the configuration ofa server shown in FIG. 1.

FIG. 3 is a flowchart showing an embodiment of the operation of theimage processing system.

FIG. 4 is a conceptual diagram showing an example of a layout in atwo-page spread page of an electronic album.

FIGS. 5A and 5B are conceptual diagrams showing an example of two imageshaving a same constitution.

FIG. 6 is a conceptual diagram showing an example of an image assignedwith pieces of common tag information “apple” and “dog”.

FIG. 7 is a conceptual diagram showing an example of a layout of a pagein an electronic album.

DETAILED DESCRIPTION OF THE INVENTION

An image processing device, an image processing method and a recordingmedium of the invention are described below in detail with reference topreferred embodiments shown in the accompanying drawings.

FIG. 1 is a block diagram showing an embodiment of the configuration ofan image processing system of the invention. The image processing system10 shown in FIG. 1 ranks a piece of tag information based on anassignment ratio of the piece of tag information assigned to imagesowned by a user. The image processing system 10 includes a server 12 andone or more clients 14 connected to the server 12 via a network 16 suchas the Internet.

FIG. 2 is a block diagram showing an embodiment of the configuration ofa server shown in FIG. 1. The server 12 shown in FIG. 2 is an example ofthe image processing device of the invention and includes an instructionacquiring section 18, an image acquiring section 20, an image analyzer22, a tag information assignor 24, a first assignment ratio calculator26, a second assignment ratio calculator 28 and a tag ranking determiner30.

In the server 12, the instruction acquiring section 18 acquires aninstruction input by a user.

The instruction acquiring section 18 acquires, from the client 14 viathe network 16, various instructions the user has input at the client14. The user's instructions include an instruction for specifying animage to be processed and an instruction for specifying a piece of taginformation.

The image acquiring section 20 acquires an image owned by the user basedon the user's instruction acquired by the instruction acquiring section18.

The image acquiring section 20 may acquire an image that is uploadedfrom the client 14 of the user to the server 12 via the network 16 inresponse to the user's instruction or acquire a second image or secondimages including at least a part of images specified in response to theuser's instruction from a first image or first images already uploadedto, i.e., already held by the server 12.

The image analyzer 22 carries out image analysis on the image acquiredby the image acquiring section 20.

The image analysis carried out by the image analyzer 22 includes, inaddition to detection of a person and a non-person object that arepresent in an image, detection of positions and areas of regions where aperson and an object lie in the image, or the like.

The tag information assignor 24 assigns each image with one or morepieces of tag information corresponding to one or more objects presentin the image based on the result of the image analysis carried out bythe image analyzer 22.

Based on the result of the image analysis, for example, when presence ofan object “apple” in an image is detected, the tag information assignor24 assigns the image with “apple” tag information. The piece of taginformation may be named the same as the object or a name different fromthe object. Moreover, an image may be assigned with a piece of taginformation having an arbitrary name specified in accordance with auser's instruction.

The first assignment ratio calculator 26 calculates an assignment ratioof each piece of common tag information assigned to and shared by imagesowned by a plurality of users as a first assignment ratio.

Further, the second assignment ratio calculator 28 calculates anassignment ratio of each piece of common tag information assigned toimages owned by each user as a second assignment ratio.

Each user registers account information including the user's name,address, user identifier (ID) and password, for example, in the imageprocessing system 10. In order to use the image processing system 10,each user inputs the user ID and the password to log in to the imageprocessing system 10. Accordingly, the image processing system 10 canidentify each user and an image owned by each user based on the accountinformation of each user.

The common tag information may be a piece of tag information that isassigned to and shared by images owned by a plurality of users inaccordance with an ordinary, non-person object present in images, suchas “ocean,” “mountain,” “apple” or “cat”. On the other hand, specifictag information (individual tag information) dedicated to each user(person) may be a piece of tag information that is assigned to an imagein accordance with a specific person or a specific object present in theimage, such as “Taro” or “Hanako”.

Further, an assignment ratio of a piece of common tag information refersto a ratio of the number of assignments of a piece of common taginformation assigned to images with respect to the number of imagesowned by a user.

For instance, the second assignment ratio calculator 28 can calculatethe second assignment ratio of each piece of common tag information foreach user by dividing the number of assignments of the piece of commontag information assigned to images owned by the user by the number ofimages owned by the user.

In addition, the first assignment ratio calculator 26 can calculate thefirst assignment ratio of each piece of common tag information bysumming up the second assignment ratios of a piece of common taginformation assigned to images owned by a plurality of users andaveraging the sum.

Subsequently, the tag ranking determiner 30 ranks each piece of commontag information assigned to images owned by each user based on adifference between the first assignment ratio and the second assignmentratio of the piece of common tag information.

For instance, the tag ranking determiner 30 can subtract the firstassignment ratio from the second assignment ratio to calculate thedifference therebetween for each piece of common tag information, andrank each piece of common tag information such that a piece of commontag information having a larger difference is higher in rankings.

The client 14 is a terminal device owned by each user, such as apersonal computer, tablet computer or smartphone. Each client 14includes an input unit such as a keyboard, a mouse or a touch panel, adisplay such as a liquid crystal display or a touch panel, and acontroller that controls processing for causing the input unit toacquire instructions input by a user, processing for causing the displayto display various pieces of information, and other processing.

The server 12 and the client 14 each have transmitting and receivingsections therein for transmitting and receiving various pieces of databetween them via the network 16, although not described in thisembodiment for avoiding the complexity.

Next, the operation of the image processing system 10 for ranking apiece of common tag information assigned to images owned by a user isdescribed with reference to the flowchart shown in FIG. 3.

First, at the client 14, a user inputs an instruction for specifying animage to be processed. The instruction is transmitted from the client 14of the user to the server 12 via the network 16.

In the server 12, the instruction acquiring section 18 acquires theuser's instruction transmitted from the client 14 of the user via thenetwork 16 (Step S1).

Based on the user's instruction acquired by the instruction acquiringsection 18, the image acquiring section 20 acquires an image or imagesowned by the user (Step S2).

Acquisition of an image or images described above is repeated for eachof a plurality of users.

Subsequently, an image owned by the user is assigned with a piece of taginformation.

For this step, first, one of the images owned by the user is extracted(Step S3).

Subsequently, the image analyzer 22 carries out image analysis on animage acquired by the image acquiring section 20 (Step S4), and the taginformation assignor 24 assigns the image with a piece of taginformation corresponding to an object present in the image based on theresult of the image analysis (Step S5).

Here, it is determined whether every one of the images owned by the useris assigned with a piece of tag information (Step S6). Accordingly, whenevery one of the images owned by the user is not assigned with a pieceof tag information (“No” at Step S6), the process returns to Step S3,and the above-described operation is repeated. On the other hand, whenevery one of the images owned by the user is assigned with a piece oftag information (“Yes” at Step S6), the process proceeds to Step S7.

Images owned by each of a plurality of users are assigned with pieces oftag information as described above.

Subsequently, the first assignment ratio and the second assignment ratioof each piece of common tag information are calculated.

For the step, first, one of the images owned by the user is extracted(Step S7).

Subsequently, the second assignment ratio calculator 28 calculates thesecond assignment ratio of each piece of common tag information for eachuser (Step S8).

Here, it is determined whether the second assignment ratio of everypiece of common tag information is calculated for each user (Step S9).When the second assignment ratio of every piece of common taginformation is not calculated (“No” at Step S9), the process returns toStep S7, and the above-described operation is repeated. On the otherhand, when the second assignment ratio of every piece of common taginformation is calculated (“Yes” at Step S9), the first assignment ratiocalculator 26 then calculates the first assignment ratio of each pieceof common tag information (Step S10).

Subsequently, a piece of common tag information is ranked.

For the step, first, the tag ranking determiner 30 subtracts the firstassignment ratio from the second assignment ratio to calculate adifference therebetween for each piece of common tag information foreach user (Step S11) and ranks the piece of common tag information basedon the difference between the assignment ratios of the piece of commontag information (Step S12).

The image processing system 10 performs the above-described imageanalysis, assignment of tag information, calculation of the first andsecond assignment ratios and determination of a rank of a piece ofcommon tag information every time a new image is added, for example.

Importance of an object corresponding to a piece of common taginformation varies depending on the user, and a rank of a piece ofcommon tag information assigned to images owned by a user showsimportance of an object for the user. In other words, by ranking a pieceof common tag information assigned to the images owned by the user,importance of an object for the user can be determined, and, further, animage including the object can be ranked.

For instance, discussed next is a case where the number of assignmentsof a piece of tag information assigned to images owned by each user iscounted, and an object corresponding to a piece of tag informationhaving a larger value of the counted number is determined to be moreimportant for the user. In this case, an object that is ordinarily seenin a daily life has a larger value of the counted number and isdetermined to be more important for the user. However, normally, anobject that is seen in a daily life is often present in images of anyuser and is not important for a particular user.

In this regard, in this embodiment, a difference between the firstassignment ratio and the second assignment ratio of a piece of commontag information is calculated, and the assignment ratio of the piece ofcommon tag information is compared between a user and other users,whereby a piece of common tag information, i.e., an object, having ahigh importance for the user can be determined. In this manner, a pieceof tag information assigned to an image owned by a user can be ranked inorder of importance for the user.

Next, the method for calculating the first assignment ratio and thesecond assignment ratio of a piece of common tag information and themethod for ranking a piece of common tag information are described withreference to an example.

TABLE 1 Common 2nd 1st Tag Assignment Assignment Information Ratio RatioDifference Rank Apple 0.8 0.3 0.5 1 Banana 0.7 0.8 −0.1 4 Football 0.60.5 0.1 3 Dog 0.3 0.1 0.2 2 Cat 0.2 0.9 −0.7 5 — — — — —

Table 1 shows the first and second assignment ratios of pieces of commontag information assigned to images owned by a user, a differenceobtained by subtracting the first assignment ratio from the secondassignment ratio, and ranks of the respective pieces of common taginformation. For example, a piece of common tag information “apple” hasthe second assignment ratio of 0.8, the first assignment ratio of 0.3, adifference obtained by subtracting the first assignment ratio from thesecond assignment ratio of 0.5. Other pieces of common tag informationare shown in the same manner.

As a piece of common tag information having a larger difference isranked to be higher, the piece of common tag information “apple” with adifference of 0.5 is ranked as the first, followed by the piece ofcommon tag information “dog” with a difference of 0.2, the piece ofcommon tag information “football” with a difference of 0.1, the piece ofcommon tag information “banana” with a difference of −0.1, and the pieceof common tag information “cat” with a difference of −0.5 being rankedas the second, third, fourth and fifth.

TABLE 2 Image (User 1) Tag Information 1 Apple Banana Taro 2 Apple Ocean3 Dog Taro 4 Mountain Dog Cat 5 Apple Apple Hanako

TABLE 3 Image (User 2) Tag Information 1 Jiro Star 2 Jiro Midori Dog 3Jiro 4 Jiro 5 Jiro

TABLE 4 Image (User 3) Tag Information 1 Saburo Baseball Bat Ball 2Apple Orange Peach 3 Ocean Dog 4 Saburo 5 Cat

Tables 2 to 4 each show pieces of tag information (common taginformation and individual tag information) assigned to five imagesowned by each of Users 1 to 3.

Image 1 of User 1 in Table 2 is assigned with pieces of tag information“apple,” “banana” and “Taro.” For example, the pieces of tag information“apple” and “banana” are pieces of common tag information, whereas thepiece of tag information “Taro” is a piece of individual taginformation. Other Images 2 to 5 are shown in the same manner. Inaddition, Images 1 to 5 of User 2 in Table 3 as well as Images 1 to 5 ofUser 3 in Table 4 are also shown in the same manner.

The piece of common tag information “apple” of User 1 is assigned onceto Image 1, once to Image 2 and twice to Image 5, totaling fourassignments. Thus, the second assignment ratio of the piece of commontag information “apple” of User 1 can be calculated by dividing thenumber of assignments of the piece of common tag information “apple”assigned to the images owned by User 1 by the number of images owned byUser 1, i.e., 4/5=0.8. Similarly, the second assignment ratio of thepiece of common tag information “apple” of User 2 is calculated as0/5=0, while the second assignment ratio of the piece of common taginformation “apple” of User 3 is calculated as 1/5=0.2.

In addition, the first assignment ratio of the piece of common taginformation “apple” assigned to the images owned by Users 1 to 3 can becalculated by summing up the second assignment ratios of the piece ofcommon tag information “apple” assigned to the images owned by Users 1to 3 and averaging the sum, i.e., (0.8+0+0.2)/3≈0.3.

Accordingly, for the piece of common tag information “apple,” thedifference between the first assignment ratio and the second assignmentratio can be found by subtracting the first assignment ratio from thesecond assignment ratio, i.e., 0.8−0.3=0.5. A difference between thefirst assignment ratio and the second assignment ratio for any otherpiece of common tag information can be calculated in the same manner.

When a single image is assigned with a same piece of common taginformation twice or more times like Image 5 in Table 2, the firstassignment ratio calculator 26 and the second assignment ratiocalculator 28 may regard the number of assignments of the same piece ofcommon tag information assigned to the single image as the number ofassignments of the piece of common tag information like in theembodiment or may regard the number of assignments of the same piece ofcommon tag information assigned to the single image as one to calculatethe first and second assignment ratios.

Next, described is an example where a rank of a piece of common taginformation is utilized to determine a rank of an image.

FIG. 4 is a conceptual diagram showing an example of a layout in atwo-page spread page of an electronic album. As illustrated in FIG. 4, aplurality of images in different sizes are arranged at various positionsvertically and horizontally in the page of an electronic album.Accordingly, the page layout sometimes has some empty spaces as shown bytwo shaded portions at the center bottom in FIG. 4. In such a case, animage selector capable of automatic layout provided to the server 12automatically selects an image, e.g., an image in which an object ispresent, from images owned by a user, and arranges the selected image inan empty space in the page layout.

In order to select an image having an object to fill an empty space in apage layout of an electronic album with the selected image in thismanner, a rank of a piece of common tag information according to theinvention can be utilized. In this case, based on the rank of a piece ofcommon tag information, the image selector can automatically select anduse one or more images in which an object with high importance for auser is present from images owned by the user. As a result, theinvention can provide a satisfying electronic album for the user.

In order to produce an electronic album consisting of a plurality ofimages, a rank of piece of common tag information according to theinvention can be also utilized. In this case, in order to select one ormore images having an object for use in an electronic album from imagesowned by a user, the image selector selects one or more images having anobject corresponding to a piece of common tag information in order fromthe highest rank toward the lowest rank of pieces of common taginformation.

Next, FIGS. 5A and 5B are conceptual diagrams showing an example of twoimages having the same constitution. Each of the two images in FIGS. 5Aand 5B shows a person on the left hand side, whereas FIG. 5A shows anapple and FIG. 5B shows a football on the right hand side. That is, thetwo images have the same constitution but are different in the object.In particular, the image in FIG. 5A has an object of an apple, and theimage in FIG. 5B has an object of a football.

When two images having the same constitution as described above, theimages can be ranked depending on the object using a rank of a piece oftag information according to the invention. In order to select one outof two images having the same constitution, the image selector canautomatically select one including an object that is highly importantfor the user, i.e., the image that is satisfying for the user.

Similarly, in order to select a certain number of images from aplurality of images having the same constitution, a rank of a piece ofcommon tag information according to the invention can be also utilized.For instance, in order to select one out of two or more images havingthe same constitution but different objects from images owned by a user,the image selector selects a single image having an object correspondingto the piece of common tag information in order from a highest ranktoward a lowest rank of pieces of common tag information.

Moreover, when an image includes two or more objects each assigned witha piece of common tag information, an image ranking determiner capableof automatic layout provided to the server 12 may rank the image basedon the ranks of pieces of common tag information assigned to imagesowned by each user.

For instance, when an image is assigned with two or more pieces ofcommon tag information, the image ranking determiner may rank the imagebased on the rank of a piece of common tag information having thehighest rank among the two or more pieces of common tag information.

For instance, as illustrated in FIG. 6, when an apple and a dog arepresent in an image, and the piece of common tag information “apple” hasthe higher rank than the piece of common tag information “dog,” theimage is ranked based on the rank of the piece of common tag information“apple.”

Moreover, when an image is assigned with two or more pieces of commontag information, the image ranking determiner may rank the image basedon the rank of a piece of common tag information corresponding to thesum of differences each between the first assignment ratio and thesecond assignment ratio of the two or more pieces of common taginformation.

In an example as illustrated in FIG. 6, the difference between the firstassignment ratio and the second assignment ratio of the piece of commontag information “apple” and the difference between the first assignmentratio and the second assignment ratio of the piece of common taginformation “dog” are calculated, and the sum of the assignment ratiodifferences of the piece of common tag information “apple” and of thepiece of common tag information “dog” is calculated. Thereafter, theimage is ranked based on the rank of a piece of common tag informationcorresponding to the sum of the assignment ratio differences thuscalculated.

Moreover, when an image is assigned with two or more pieces of commontag information, the image ranking determiner may rank the image basedon the rank of a piece of common tag information corresponding to thesum of differences each between the first assignment ratio and thesecond assignment ratio of each of the two or more pieces of common taginformation, the differences being weighted in accordance with areas oftwo or more objects corresponding to the two or more pieces of commontag information.

In the example as illustrated in FIG. 6, the area of the “apple” regionand the area of “dog” region each occupy about 1/4 of the whole area ofthe image region. In this case, differences each between the firstassignment ratio and the second assignment ratio of each of the piecesof common tag information “apple” and “dog” are weighted by 1/4, and thesum of the weighted differences each between the first assignment ratioand the second assignment ratio of each of the pieces of common taginformation “apple” and “dog” is calculated. Thereafter, the image isranked based on the rank of a piece of common tag informationcorresponding to the sum of differences thus calculated.

Moreover, when the image ranking determiner ranked an image based on therank of a piece of common tag information, the image selector may selectone or more images from images owned by a user based on the rank of animage.

The ranking of pieces of common tag information according to theinvention can be adopted not only when one or more images areautomatically selected from images owned by a user but also when allimages owned by a user are used for producing an electronic album, forexample.

FIG. 7 is a conceptual diagram showing an example of a layout of a pagein an electronic album. In the layout illustrated in FIG. 7, images ofan apple, a dog and a football are shown, gradually decreasing in sizein this order.

When arranging images on a page in an electronic album, sizes of theimages can be determined in accordance with importance of each image byutilizing the ranking of pieces of tag information according to theinvention. In this case, an image layout section capable of automaticlayout provided to the server 12 can produce a satisfying electronicalbum for a user by arranging images such that, when only objects areused to produce an electronic album, an image including an object withhigher importance for the user is made larger in size and an imageincluding an object with lower importance for the user is made smallerin size.

In this case, the image layout section arranges two or more imageshaving objects in a page of an electronic album such that the size of animage with an object corresponding to a piece of common tag informationhaving a higher rank is larger than the size of an image with an objectcorresponding to a piece of common tag information having a lower rank.

The tag information assignor 24 may select, from a certain number ofpieces of tag information preliminarily set in the image processingsystem 10, a piece of tag information corresponding to an object presentin an image and assign the image with the corresponding piece of taginformation. Alternatively, an image may be assigned with a piece of taginformation corresponding to an object present in the image, the pieceof tag information being chosen from pieces of tag informationdesignated from the outside in accordance with a user's instruction, inaddition to a certain number of pieces of tag information preliminarilyset.

It is not essential that the tag information assignor 24 assigns animage with a piece of tag information, but an image that has beenpreliminarily assigned with a piece of tag information may be used.

Like in the above-described embodiment, since the second assignmentratios of pieces of common tag information assigned to images owned by aplurality of users are summed and averaged to calculate the firstassignment ratios, influences from a user owning an enormously largenumber of images compared to other users can be excluded.

However, in a case where tremendous users are involved, the total numberof images owned by the users becomes so huge that an influence of thenumber of images owned by each user can be ignored. Therefore, the firstassignment ratio calculator 26 may calculate the first assignment ratiofor each piece of common tag information by dividing the number ofassignments of a piece of common tag information assigned to imagesowned by a plurality of users by the number of images owned by theplurality of users.

For instance, while a user's preference may change with time, assumingthat the user's latest preference should be more important for the user,it can be understood that an image of the user's latest preference hasthe higher importance than an image of the user's past preference.Accordingly, the first assignment ratio calculator 26 and the secondassignment ratio calculator 28 may calculate the first assignment ratioand the second assignment ratio by weighting the number of assignmentsof each piece of common tag information such that the weight becomessmaller from the latest toward the past.

In addition, while a plurality of users may be all of the usersregistered in the image processing system 10, the plurality of users maybe, for example, those in at least one of the groups categorized bycountry, by prefecture, by age and by gender, based on the accountinformation on the respective users. By having users categorized intogroups, users having similar tendency can be gathered into a singlegroup, so a piece of common tag information can be ranked moreprecisely.

Although all functions corresponding to the image processing device ofthe invention are provided to the server 12 in the above-describedembodiment, the invention is not limited thereto, and a part of or allof the functions may be provided to the client 14.

The device of the invention may be composed of constituent elements thatare dedicated hardware devices or composed of a computer in whichoperations of the constituent elements are programmed.

The method of the invention can be carried out by, for instance, aprogram that causes a computer to implement steps of the method. Acomputer readable recording medium having the program recorded thereonmay also be provided.

While the invention has been described above in detail, the invention isnot limited to the above embodiment, and various improvements andmodifications may be made without departing from the spirit and scope ofthe invention.

What is claimed is:
 1. An image processing device comprising: an image analyzer configured to carry out image analysis on an image; a tag information assignor configured to assign the image with one or more pieces of tag information corresponding to one or more objects present in the image based on a result of the image analysis; a first assignment ratio calculator configured to calculate an assignment ratio of each piece of common tag information assigned to images owned by a plurality of users, which is a ratio of a number of assignments of the each piece of common tag information assigned to images owned by the plurality of users with respect to a number of images owned by the plurality of users, as a first assignment ratio, the each piece of common tag information being a piece of tag information assigned to and shared by images owned by the plurality of users; a second assignment ratio calculator configured to calculate an assignment ratio of the each piece of common tag information assigned to images owned by each user, which is a ratio of a number of assignments of the each piece of common tag information assigned to images owned by the each user with respect to a number of images owned by the each user, as a second assignment ratio; a tag ranking determiner configured to rank the each piece of common tag information assigned to images owned by the each user based on a difference between the first assignment ratio and the second assignment ratio of the each piece of common tag information; an image selector configured to select one or more images from images owned by a user based on a rank of the each piece of common tag information; and an image layout section configured to generate a synthetic image of the selected one or more images, wherein the image analyzer, the tag information assignor, the first assignment ratio calculator, the second assignment ratio calculator, the tag ranking determiner, the image selector and the image layout section are composed of hardware or a programmed computer.
 2. The image processing device according to claim 1, wherein the tag ranking determiner subtracts the first assignment ratio from the second assignment ratio to calculate a difference therebetween for the each piece of common tag information and ranks the each piece of common tag information such that a piece of common tag information having a larger difference is higher in rankings.
 3. The image processing device according to claim 1, wherein the second assignment ratio calculator calculates, as the second assignment ratio of the each piece of common tag information for the each user, a ratio of dividing a number of assignments of the each piece of common tag information assigned to images owned by the each user by a number of images owned by the each user.
 4. The image processing device according to claim 3, wherein the first assignment ratio calculator calculates the first assignment ratio of the each piece of common tag information by summing up second assignment ratios of the each piece of common tag information assigned to images owned by the plurality of users and averaging a sum of the second assignment ratios.
 5. The image processing device according to claim 3, wherein the first assignment ratio calculator calculates, as the first assignment ratio for the each piece of common tag information, a ratio of dividing a number of assignments of the each piece of common tag information assigned to images owned by the plurality of users by a number of images owned by the plurality of users.
 6. The image processing device according to claim 3, wherein, when a single image is assigned with a same piece of common tag information twice or more times, the first assignment ratio calculator and the second assignment ratio calculator regard a number of assignments of the same piece of common tag information assigned to the single image as a number of assignments of the piece of common tag information to calculate the first assignment ratio and the second assignment ratio of the piece of common tag information.
 7. The image processing device according to claim 3, wherein, when a single image is assigned with a same piece of common tag information twice or more times, the first assignment ratio calculator and the second assignment ratio calculator regard a number of assignments of the same piece of common tag information assigned to the single image as one to calculate the first assignment ratio and the second assignment ratio of the piece of common tag information.
 8. The image processing device according to claim 3, wherein the first assignment ratio calculator and the second assignment ratio calculator calculate the first assignment ratio and the second assignment ratio by weighting a number of assignments of the each piece of common tag information such that a weight becomes smaller from the latest toward the past.
 9. The image processing device according to claim 1, wherein, in order to select one or more images having the one or more objects for use in the synthetic image from images owned by a user, the image selector selects one or more images having an object corresponding to a piece of common tag information in order from the highest rank toward the lowest rank of pieces of common tag information.
 10. The image processing device according to claim 1, wherein, in order to select one out of two or more images having a same constitution but different objects from images owned by a user, the image selector selects a single image having an object corresponding to a piece of common tag information in order from a highest rank toward a lowest rank of pieces of common tag information.
 11. The image processing device according to claim 1, further comprising an image ranking determiner configured to rank an image based on a rank of the each piece of common tag information assigned to images owned by the each user, wherein the image ranking determiner is composed of hardware or a programmed computer.
 12. The image processing device according to claim 11, wherein, when an image is assigned with two or more pieces of common tag information, the image ranking determiner ranks the image based on a rank of a piece of common tag information having a highest rank among the two or more pieces of common tag information.
 13. The image processing device according to claim 12, wherein the image selector is further configured to select one or more images from images owned by the user based on a rank of the image.
 14. The image processing device according to claim 11, wherein, when an image is assigned with two or more pieces of common tag information, the image ranking determiner ranks the image based on a rank of a piece of common tag information corresponding to a sum of differences each between the first assignment ratio and the second assignment ratio of each of the two or more pieces of common tag information.
 15. The image processing device according to claim 11, wherein, when an image is assigned with two or more pieces of common tag information, the image ranking determiner ranks the image based on a rank of a piece of common tag information corresponding to a sum of differences each between the first assignment ratio and the second assignment ratio of each of the two or more pieces of common tag information, the differences between the first assignment ratio and the second assignment ratio of each of the two or more pieces of common tag information being weighted in accordance with areas of two or more objects corresponding to the two or more pieces of common tag information.
 16. The image processing device according to claim 1, wherein the image layout section is further configured to arrange two or more images each having the object in the synthetic image such that a size of an image with an object corresponding to a piece of common tag information having a higher rank is larger than a size of an image with an object corresponding to a piece of common tag information having a lower rank.
 17. The image processing device according to claim 1, wherein the image processing device performs image analysis, assignment of the tag information, calculation of the first assignment ratio and the second assignment ratio and determination of a rank of a piece of common tag information every time a new image is added.
 18. An image processing method comprising: carrying out image analysis on an image, with an image analyzer; assigning the image with one or more pieces of tag information corresponding to one or more objects present in the image based on a result of the image analysis, with a tag information assignor; calculating an assignment ratio of each piece of common tag information assigned to images owned by a plurality of users, which is a ratio of a number of assignments of the each piece of common tag information assigned to images owned by the plurality of users with respect to a number of images owned by the plurality of users, as a first assignment ratio, with a first assignment ratio calculator, the each piece of common tag information being a piece of tag information assigned to and shared by images owned by the plurality of users; calculating an assignment ratio of the each piece of common tag information assigned to images owned by each user, which is a ratio of a number of assignments of the each piece of common tag information assigned to images owned by the each user with respect to a number of images owned by the each user, as a second assignment ratio, with a second assignment ratio calculator; ranking the each piece of common tag information assigned to images owned by each user based on a difference between the first assignment ratio and the second assignment ratio of the each piece of common tag information, with a tag ranking determiner; selecting one or more images from images owned by a user based on a rank of the each piece of common tag information, with an image selector; and generating a synthetic image of the selected one or more images, with an image layout section, wherein the image analyzer, the tag information assignor, the first assignment ratio calculator, the second assignment ratio calculator, the tag ranking determiner, the image selector and the image layout section are composed of hardware or a programmed computer.
 19. A non-transitory computer readable recording medium having recorded thereon a program for causing a computer to execute each step of the image processing method according to claim
 18. 